A number of products and methods have been developed for producing 3-D images from two-dimensional images. Steenblik in U.S. Pat. Nos. 4,597,634, 4,717,239, and 5,002,364 teaches the use of diffractive optical elements with double prisms, one prism being made of a low-dispersion prism and the second prism being made of a high-dispersion prism. Takahaski, et al in U.S. Pat. No. 5,144,344 teaches the use of spectacles based on the Pulfrich effect with light filtering lens of different optical densities. Beard in U.S. Pat. No. 4,705,371 teaches the use of gradients of optical densities going from the center to the periphery of a lens.
Hirano in U.S. Pat. No. 4,429,951 teaches the use of spectacles with lenses that can rotate about a vertical axis to create stereoscopic effects. Laden in U.S. Pat. No. 4,049,339 teaches the use of spectacles with opaque temples and an opaque rectangular frame, except for triangular shaped lenses positioned in the frame adjacent to a nosepiece.
Davino, U.S. Pat. No. 6,598,968, ‘3-Dimensional Movie and Television Viewer’, teaches an opaque frame that can be placed in front of a user's eyes like a pair of glasses for 3-D viewing to take advantage of the Pulfrich effect. The frame has two rectangular apertures. These apertures are spaced to be in directly in front of the user's eyes. One aperture is empty; the other opening has plural vertical strips, preferably two, made of polyester film. Between the outer edge of the aperture and the outermost vertical strip is diffractive optical material. The surface of the strips facing away from the person's face might be painted black. Images from a television set or a movie screen appear three dimensional when viewed through the frame with both eyes open.
Dones, U.S. Pat. No. 4,805,988, ‘Personal Viewing Video Device’, teaches a personal video viewing device which allows the simultaneous viewing of a stereoscopic external image as well as a monoscopic electronic image. This is accomplished using two optical systems which share particular components. The relative intensity of both images may be adjusted using a three-iris system where each iris may be a mechanical diaphragm, an electronically controlled liquid crystal device, or a pair of polarized discs whose relative rotational orientation controls the transmissivity of the disc pair.
Beard in U.S. Pat. No. 4,893,898 teaches a method for creating a 3-D television effect in which a scene is recorded with a relative lateral movement between the scene and the recording mechanism. The recording is played back and viewed through a pair of viewer glasses in which one of the lenses is darker and has a spectral transmission characterized by a reduced transmissivity in at least one, and preferably all three, of the television's peak radiant energy wavebands. The lighter lens, on the other hand, has a spectral transmission characterized by a reduced transmissivity at wavelengths removed from the television energy peaks. The result is a substantially greater effective optical density differential between the two lenses when viewing television than in normal ambient light. This produces a very noticeable 3-D effect for television scenes with the proper movement, while avoiding the prior “dead eye” effect associated with too great a density differential in ordinary light. Further enhancement is achieved by providing the darker lens with a higher transmissivity in the blue and red regions than in the yellow or green regions.
Other patents deal with image processing to measure motion in a moving picture and include Iue U.S. Pat. No. 5,717,415, Nagaya U.S. Pat. No. 5,721,692 and Gerard De Haan U.S. Pat. No. 6,385,245.
Iue in U.S. Pat. No. 5,717,415 teaches a method of converting two-dimensional images into three-dimensional images. A right eye image signal and a left eye image signal between which there is relatively a time difference or a luminance difference are produced from a two-dimensional image signal, thereby to convert two-dimensional images into three-dimensional images.
In U.S. Pat. No. 5,721,692, Nagaya et al present a ‘Moving Object Detection Apparatus’. In that disclosed invention, a moving object is detected from a movie that has a complicated background. In order to detect the moving object, there is provided a unit for inputting the movie, a display unit for outputting a processed result, a unit for judging an interval which is predicted to belong to the background as part of a pixel region in the movie, a unit for extracting the moving object and a unit for calculating the moving direction and velocity of the moving object. Even with a complicated background in which not only a change in illumination condition, but also a change in structure occurs, the presence of the structure change of the background can be determined so as to detect and/or extract the moving object in real time. Additionally, the moving direction and velocity of the moving object can be determined.
De Haan U.S. Pat. No. 6,385,245 teaches a method of estimating motion in which at least two motion parameter sets are generated from input video data. A motion parameter set is a set of parameters describing motion in an image, and by means of which motion can be calculated.
Visual effects are important in motion pictures and have the potential to expand the viewing enjoyment of moviegoers. For example the movement effect ‘Bullet Time’ utilized in the movie ‘The Matrix’ was critical to the appeal of the movie.
Visual effects for 3-dimensional motion pictures include such motion pictures as ‘Charge at Feather River’, starring Guy Madison. The ‘Vincent Price movie ‘House of Wax’ was originally released as a 3-D thriller. The 3-D movie fad of the early to mid-1950s however soon faded due to complexity of the technologies and potential for improper synchronization, and misalignment of left and right eye images as delivered to the viewer.
TV 3-D motion pictures have been attempted from time-to-time. Theatric Support produced the first TV Pulfrich event in 1989 for Fox Television—“The Rose Parade in 3D Live.” In order to sustain the illusion of realistic depth these 3-D Pulfrich effect TV shows require all foreground screen action to move in one consistent direction, matched to the fixed light-diminishing lens of special spectacles provided to viewers for each broadcast. This enormous constraint (for all screen action to proceed in one direction) placed on the producers of the motion picture is due to the realistic expectation that viewers were not going to invert their spectacles so as to switch the light-diminishing filter from one eye to another for each change in screen-action direction. For the great majority of viewers the limitation of spectacles with a fixed filter, either left or right, meant the 3D effect would be available only with movies produced specifically for that viewing spectacles design.
With the exception of Sony I-max 3-D presentations, which require special theater/screening facilities unique to the requirements of I-Max technology, 3-dimensional motion pictures remain a novelty. Despite the wide appeal to viewers, the difficulties and burden on motion picture producers, distributors, TV networks, motion picture theaters, and on the viewers has been a barrier to their wide scale acceptance. Because of recent loss of theater revenue to the web an effort is again underway to offer 3-D cinema that audiences will come out for, results still to be determined. The new offerings will be digital, and so many problems having to do with film may be bypassed. TV networks are also attempting to find a means to distribute 3D to their viewers.
The following background information is provided for a better understanding of the present invention:
The Human Eye and Depth Perception
The human eye can sense and interpret electromagnetic radiation in the wavelengths of about 400 to 700 nanometers—visual light to the human eye. Many electronic instruments, such as camcorders, cell phone cameras, etc., are also able to sense and record electromagnetic radiation in the band of wavelengths 400-700 nanometer.
To facilitate vision, the human eye does considerable ‘image processing’ before the brain gets the image.
When light ceases to stimulate the eyes photoreceptors, the photoreceptors continue to send signals, or ‘fire’ for a fraction of a second afterwards. This is called ‘persistence of vision’, and is key to the invention of motion pictures that allows humans to perceive rapidly changing and flickering individual images as a continuous moving image.
The photoreceptors of the human eye do not ‘fire’ instantaneously. Low light conditions can take a few thousands of a second longer to transmit signals than under higher light conditions. Causing less light to be received in one eye than another eye, thus causing the photoreceptors of the right and left eyes to transmit their ‘pictures’ at slightly different times, explains in part the Pulfrich 3-D illusion, which is utilized in the invention of the 3Deeps system. This is also cause of what is commonly referred to as ‘night vision’.
Once signals are sent to the eyes, the brain processes the dual images together (images received from the left and right eye) presenting the world to the mind in 3-dimensions or with ‘Depth Perception’. This is accomplished by several means that have been long understood.
Stereopsis is the primary means of depth perception and requires sight from both eyes. The brain processes the dual images, and triangulates the two images received from the left and right eye, sensing how far inward the eyes are pointing to focus the object.
Perspective uses information that if two objects are the same size, but one object is closer to the viewer than the other object, then the closer object will appear larger. The brain processes this information to provide clues that are interpreted as perceived depth.
Motion parallax is the effect that the further objects are away from us, the slower they move across our field of vision. The brain processes motion parallax information to provide clues that are interpreted as perceived depth.
Shadows provide another clue to the human brain, which can be perceived as depth. Shading objects, to create the illusions of shadows and thus depth, is widely used in illustration to imply depth without actually penetrating (perceptually) the 2-D screen surface.
Methods of Producing 3-D Illusion in Moving Pictures
Motion pictures are images in 2-dimensions. However, several methods have been developed for providing the illusion of depth in motion pictures. These include the Anaglyph, IMAX (Polaroid) and Pulfrich 3-dimensional illusions.
Anaglyph 3-Dimensional Illusion
“Anaglyph” refers to the red/blue or red/green glasses that are used in comic books and in cereal packets etc. The glasses consist of nothing more than one piece of transparent blue plastic and one piece of transparent red plastic. These glasses are easy to manufacture and have been around since the 1920s.
An anaglyph stereo picture starts as a normal stereo pair of images, two images of the same scene, shot from slightly different positions. One image is then made all green/blue and the other is made all red, the two are then seen together.
When the image is viewed through the glasses the red parts are seen by one eye and the other sees the green/blue parts. This effect is fairly simple to do with photography, and extremely easy to do on a PC, and it can even be hand-drawn. The main limitation of this technique is that because the color is used in this way, the true color content of the image is usually lost and the resulting images are usually in black and white. As the colors compete for dominance they may appear unstable and monochromatic. A few images can retain a resemblance to their original color content, but the photographer has to be very selective with color and picture content.
IMAX (Polaroid) 3-Dimensional Illusion
IMAX creates the illusion of 3-dimensional depth by recording the motion pictures on two separate rolls of film with two camera lenses to represent the left and right eyes. These lenses are separated by an interocular distance of about 2.5 in., the average distance between a human's eyes. By recording on two separate rolls of film for the left and right eyes, and then projecting them simultaneously, IMAX can create a 3-Dimensional illusion for viewers.
IMAX uses either of two different methods to create the 3D illusion in the theatre. The first method relies on polarization. During projection, the left eye image is polarized in one direction and the right eye image polarized perpendicular to the left eye image as they are projected on the IMAX screen. By wearing special viewing glasses with lenses polarized in their respective directions to match the projection, the left eye image can be viewed only by the left eye since the polarization of the left lens will cancel out that of the right eye projection, and the right eye image can be viewed only by the right eye since the polarization of the right lens will cancel out that of the left eye projection.
IMAX also uses another method—shutter glasses—for 3D viewing. This method of 3D projection involves the use of LCD shutter glasses that use similarly polarized lenses for both eyes. The left and right eye images are projected on the viewing screen in alternate frames. These LCD shutter glasses are synchronized to the projector. The projector displays the left and right images that are momentarily viewed by the appropriate eye by allowing that LCD lens to become transparent while the other remains opaque. That is when the left eye frame is projected on the screen, the left lens of the shutter glasses becomes transparent and the right lens of the shutter glasses becomes opaque. When the next frame is projected on the screen—a frame for the right eye—the left lens becomes opaque and the right lens becomes transparent.
In both the IMAX 3D systems only the correct eye is allowed to view the correct image while the other eye is ‘blinded’. The ‘transparent’ state is actually quite dark, and occludes about 35% of the projected light to the viewing eye while the non-viewing eye is supposed to view no image at all.
Problems with 3-D Motion Pictures
With the exception of Sony I-Max 3-d, a special cine-technology requiring theaters designed for its screening requirements, 3Dimensional motion pictures have never caught on, except as a short-term fad, because a myriad of problems continue to make 3-dimensional motion pictures unacceptable to producers and viewers of motion pictures. Despite concerted efforts, 3-dimensional motion pictures continue to be nothing more than a novelty. There are many problems and constraints involving the production, projection, and viewing of 3-dimensional motion pictures.
Production: The commonly used anaglyph 3-dimensional movie systems require special cameras that have dual lenses, and capture 2-images on each frame. To have a version of the motion picture that can be viewed without special glasses requires that a separate version of the motion picture be shot with a regular camera so there is only one image per video frame and not simply the selection of one or the other perspective. Similarly, IMAX and shutter glass systems require special cameras and processing with separate versions of the motion picture for 2D and 3D viewing. Filming movies in 3D add as much as $10 million dollars to production costs, it has been reported.
Projection: Some 3-dimensional systems require the synchronization and projection by more than 2 cameras in order to achieve the effect. “Hitachi, Ltd has developed a 3D display called Transpost 3D which can be viewed from any direction without wearing special glasses, and utilize twelve cameras and rotating display that allow Transpost 3D motion pictures that can be seen to appear as floating in the display. The principle of the device is that 2D images of an object taken from 24 different directions are projected to a special rotating screen. On a large scale this is commercially unfeasible, as special effects in a motion picture must be able to be projected with standard projection equipment in a movie theater, TV or other broadcast equipment.
Viewing: As a commercial requirement, any special effect in a motion picture must allow viewing on a movie screen, and other viewing venues such as TV, DVD, VCR, PC computer screen, plasma and LCD displays. From the viewer's vantage, 3-dimensional glasses, whether anaglyph glasses or Pulfrich glasses, which are used in the majority of 3-dimensional efforts, if poorly made or worn incorrectly are uncomfortable and may cause undue eyestrain or headaches. Experiencing such headache motivates people to shy away from 3-D motion pictures.
Because of these and other problems, 3-dimensional motion pictures have never been more than a novelty. The inconvenience and cost factors for producers, special equipment projection requirements, and viewer discomfort raise a sufficiently high barrier to 3-dimensional motion pictures that they are rarely produced. A main object of this invention is to overcome these problems and constraints.
Attempts to Overcome the Problems of 3-D Motion Pictures
Different formulations of shutter glasses have been implemented over the last few decades, but without much large-scale commercial success. A shutter glasses solution generally require two images for each image of video, with shutter covering or uncovering each eye of the viewer. This allows one eye to see, than the other, with the shutters timed and synchronized with the video so that each eye only sees the image intended for it. Recent advances have eliminated mechanical shutter, and now use lens that darken when an electric current is passed through it.
Some shutter glass systems are wired to a control device while some shutter glass systems use wireless infrared signaling to control the state of the lenses.
CrystalEyes is the name of a stereoscopic viewing product produced by the StereoGraphics Corporation of San Rafael, Calif. They are lightweight, wireless liquid crystal shuttering eyewear that are used to allow the user to view alternating field sequential stereo images. The source of the images alternately displays a left-eye view followed by a right-eye view. CrystalEyes' shutters can block either of the user's eyes so that only images appropriate for each eye are allowed to pass. A wireless infrared communications link synchronizes the shuttering of the eyewear to the images displayed on the monitor or other viewing screen. CrystalEyes shutter glasses, weight only 3.3 ounces, use two 3V lithium/manganese dioxide batteries, and have a battery life of 250 hours. This demonstrates the robustness and potential of a viewer glass solution.
Because shutter glasses only expose each eye to every other frame, the refresh rate of the video is effectively cut in half. On a TV with refresh rates of 30 frames per second (for an NTSC TV) or 25 frames per second (for a PAL TV), this is hard on the eyes because of the continual flicker. This problem is eliminated with higher refresh rates, such as on PC monitors.
However, shutter systems have not been overwhelmingly commercially successful. Motion pictures that use such stereo shutter systems require two frames for each frame of regular film. Motion pictures would then have to be produced in at least 2 versions. Also, except on high refresh rate systems, such as expensive PC monitors, the viewer sees too much ‘flicker’ causing distraction and annoyance. An additional requirement and burden is the wired or wireless signaling to control the state of the lens. LCD screens that are used on laptops generally do not have high enough refresh rates for stereoscopic shutter 3D systems. Shutter systems generally do not work well with LCD or movie projectors.
Previous versions of 3Deeps Filter Spectacles have been described in co-pending patent applications and provide a system by which ordinary 2-dimensional motion pictures can be viewed in part as a 3-dimensional motion pictures. They however were a sub-optimal solution. In the presence of screen motion, they only developed 3D from a 2D movie by a difference in optical density between the right and left lens—but did not describe any objective optimal target for those optical densities. Neither did the previous version or 3Deeps Filter spectacles address optimization of the spectacles to account for the materials from which the lenses are fabricated.
3Deeps Filter Spectacles that incorporate such double optimization are called Continuous Adjustable 3Deeps Filter Spectacles.
Continuous Adjustable 3Deeps Filter Spectacles doubly optimize 3Deeps Filter Spectacles to maximize the target optical densities of the lenses, and to account for the lens material. Double optimization of the 3Deeps Filter Spectacles has substantial benefits and Continuous Adjustable 3Deeps Filter Spectacles solves substantial problems that 3Deeps Filter Spectacles could not address.
Various embodiments described herein provide significant advantages and benefits by providing a system to view 3-dimensional and other special effects in motion pictures. Certain embodiments provide a system by which ordinary 2-dimensional motion pictures can be viewed in part as a 3-dimensional experience.
Movies are generally made from a series of single, non-repetitive pictures which are viewed at a speed that provides the viewer with the appearance of continuous movement. These series of single pictures are positioned in adjacent picture frames, in sequential order, wherein adjacent pictures are substantially similar to each other and vary only slightly from each other, Usually, movies are created using movie cameras, which capture the actual movement of the object; with animated movies, a series of individual pictures or cells are created, usually by hand or computer, and assembled in sequential order where adjacent pictures of a scene are substantially similar to each other and vary only slightly. Standard film projection is 24 frames per second, American video standard NTSC is 30 f.p.s.
The appearance of continuous movement, using only two substantially similar pictures, has been accomplished in live performance by simultaneous projection of both images onto a screen, wherein one picture may be slightly off-set from the other picture as they appear on the screen, and by rotating a two-bladed propeller, wherein the propeller blades are set off from one another by ISO degrees, in front of and between the two projectors such that the two images are made to both alternate and overlap in their appearances, with both images in turn alternating with an interval of complete darkness onscreen when both projections are blocked by the spinning propeller. A viewer, using no special spectacles or visual aids, perceives a scene of limited action (with a degree of illusionary depth) that can be sustained indefinitely in any chosen direction: an evolving yet limited action appears to be happening continually without visible return-and-start-over repetition. Thus the viewer sees a visual illusion of an event impossible in actual life. Similarly, the manner in which things appear in depth are likely to be at odds, often extremely so, with the spatial character of the original photographed scene. Further, the character of movement and of depth has been made malleable in the hands of the projectionist during performance (so much so that such film-performance has been likened to a form of puppetry); the physical shifting of one of the two projections changes the visual relationship between them and thereby the character of the screen event produced. Similarly, small changes during performance in speed, placement and direction of propeller spin will cause radical changes in the visual event produced onscreen.
Other visual arts which relate to the present invention are the Pulfrich filter. For one program, titled “Bitemporal Vision: The Sea”, viewers were invited to place a Pulfrich light-reducing filter before one eye to both enhance and transform the already apparent depth character of the presentation.
Limited to presentation in live performance, such unique visual phenomena as described has been transient theater. Attempts to capture the phenomena by way of video-camera recording of the screen-image have been disappointingly compromised, so that—in over 25 years of such presentation (of so-called “Nervous System Film Performances”) no attempt has been made to commercialize such recordings.